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ABSTRACT 


This  project  presents  contemporary  thoughts  on  the  merits  of  the  implementation  and 
effectiveness  of  a  new  Purchase  Order  automation  system.  This  case  study  reviews  the 
steps  taken  to  implement  this  system  and  the  lessons  learned  along  the  way.  Information 
is  presented  on  the  conditions,  methods,  objectives,  and  problems  associated  with  this 
project.  Information  is  also  be  presented  on  the  insights,  advantages,  and  utility  gained  in 
the  implementation  of  this  automated  system. 

The  completion  of  the  project  resulted  in  the  accomplishment  of  many  things,  they 
are  loosely  grouped  into  two  categories.  The  first  group  of  benefits  centers  around  the 
topic  of  information.  This  project,  which  implemented  a  network-based  purchase  order 
tracking  system  with  a  database,  has  dramatically  increased  the  amount  of  useful 
information  for  the  organization.  This  is  new,  and  was  not  possible  with  the  old  system. 

The  second  group  of  benefits  centers  around  control.  The  old  process  suffered  from 
a  general  lack  of  control  and  this  project,  for  the  most  part,  has  alleviated  this  problem. 
This  consequent  efficiency,  and  the  previously  mentioned  wealth  of  information  has  been 
the  result  of  the  implementation  of  a  purchase  order  tracking  system  that  is  clearly 
superior  to  its  predecessor.  This  has  been  done  to  the  benefit  and  satisfaction  of  the 
manager,  the  customers  of  the  process,  and  to  the  organization. 


iv 


THE  GRADUATE  MANAGEMENT  PROJECT 

A  Qualitative  Case  Study  on  the  Implementation  and  Effectiveness 
of  a  Purchase  Order  Automation  System 

This  project  will  present  contemporary  thoughts  on  the  merits  of  the 
implementation  and  effectiveness  of  a  new  Purchase  Order  automation  system. 
Information  will  be  presented  on  the  conditions,  methods,  objectives,  and  problems 
associated  with  this  project.  Information  will  further  be  presented  on  the  insights, 
advantages,  and  utility  gained  in  the  implementation  of  this  automated  system.  The  great 
depth  of  this  topic,  the  amount  of  information  involved  in  the  implementation,  and  the 
limited  format  of  this  paper,  has  required  the  consolidation  of  the  major  elements. 
Therefore,  I  have  divided  this  paper  into  four  sections  as  follows:  introduction,  methods 
and  procedures,  discussions,  and  conclusions. 
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INTRODUCTION 


Conditions  Which  Prompted  the  Study 

This  effort  was  initiated  by  the  George  Washington  University  Health  Plan's 
Director  for  Administration.  He  had,  for  sometime,  been  considering  automating  the 
Plan's  Purchasing  Department  because  he  felt  there  were  inadequate  management  controls 
and  no  useful  historical  information. 

I  was  originally  made  aware  of  the  director's  concerns  during  my  rotation  through 
the  Purchasing  Department  (as  part  of  my  training).  It  was  at  this  time  that  he  asked  me 
to  investigate  the  possibilities  of  automating  the  current  process.  A  summary  of  the 
operation  and  introductions  to  the  interested  parties  were  given.  Follow-on  presentations 
on  the  deficiencies  of  the  current  system  were  held  and  the  key  issues  were  identified. 

The  Director  for  Administration  had  two  key  concerns.  The  first,  and  most 
important,  dealt  with  the  lack  of  useful  historical  information.  The  current  system  was 
entirely  paper  driven,  relying  on  4-part  forms,  clearly  deficient  log  books,  and  an 
unkempt  and  unreliable  filing  system.  It  could  best  be  described  as  no  system  at  all. 

Tools  that  lend  themselves  to  this  type  of  process  and  that  are  considered  common 
(Personal  computers  (PC),  printers,  and  databases),  were  not  being  utilized.  The  system 
that  was  in  place  was  barely  adequate  for  the  day-to-day  operations,  let  alone  the  detailed 
retrospective  analysis  that  was  desired. 

Second,  there  was  a  general  lack  of  control.  There  was  no  written  policy,  no 
documented  processes,  and  no  defined  responsibilities.  The  whole  operation  was  run  by 


2 


a  "rule  of  thumb,"  and  "that's  the  way  we've  always  done  it"  mentality.  This  type  of 
environment,  as  you  could  imagine,  created  many  problems  (identical  recurring  mistakes, 
lost  purchase  orders,  and  extremely  untimely  delays).  This  combined  with  the  external 
constraints  of  the  plan  (centralized  signature  approval,  and  no  detailed  departmental 
budgets)  created  an  inefficient,  unstructured,  and  frustrating  process  that  was 
unquestionably  in  need  for  a  change.  It  was  the  combination  of  this  need  for  change,  the 
desire  for  more  information,  and  the  request  of  the  Director  of  Administration  that 
prompted  this  study. 


Statement  of  the  Problem  and  Question 

The  George  Washington  University  Health  Plan's  manual  purchase  order  tracking 
system  was  ineffective,  inefficient,  and  in  a  constant  state  of  discrepancy.  The  research 
question  that  was  examined  is:  "What  would  be  required  to  automate  the  Purchase  Order 
process  to  improve  its  functionality,  proficiency,  and  customer  service?" 
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Objectives  of  the  Study 

The  purpose  of  this  study  was  to  describe  the  process  of  implementing  a  Purchase 
Order  automation  system  and  to  evaluate  the  success  of  this  implementation.  The  many 
objectives  that  were  associated  with  this  project  are  listed  as  follows: 

•  Analyze  and  learn  the  Plan's  Purchase  Order  System 

•  Evaluate  and  select  a  software  vendor 

•  Coordinate  and  resolve  all  implementation  issues  with  vendor 

•  Develop  and  implement  a  training  plan 

•  Develop  new  policy  and  procedures  for  Purchase  Order  System 

•  Determine  the  opinion  and  satisfaction  of  the  current  system 

•  Deploy  the  new  system 

•  Evaluate  implementation  of  the  new  system 


Literature  Review 

The  unique  nature  of  this  specific  project  eliminated  the  availability  of  other 
analogous  cases  to  build  upon.  However,  it  did  not  eliminate  instances  of  similar  ones. 
There  were  some  that  identified,  described,  and  even  recommended  various  approaches 
that  were  applicable  to  this  situation  and  are  worth  presenting. 

The  first  such  case  dealt  with  the  implementation  and  evaluation  of  a  new  computer 
system.  In  this  study,  qualitative  and  quantitative  methods  were  combined  in  a 
longitudinal  study  of  interrelationships  between  perceptions  of  work  and  a  computer 
information  system  (Kaplan  1988).  It  was  based  on  a  study  of  the  introduction  of 
computer  technology  in  a  medical  laboratory  at  a  university  medical  center.  The 
problems  and  contributions  stemming  from  different  methodological  approaches  were 
examined,  and  the  results  suggested  merit  in  investigating  specifically  how  a  new 
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implementation  of  a  computer  system  may  affect  users’  jobs  (Kaplan  1988).  Although 
this  implementation  was  a  computer  system  in  the  traditional  sense,  and  did  not  closely 
resemble  the  automation  effort  that  I  was  working  on,  there  was  still  one  relevant  aspect. 
The  case  study  used  a  qualitative  longitudinal  evaluation  to  judge  the  implementation  of 
the  system  over  time  (this  case  study  uses  a  similar  method). 

Another  fitting  case  study  presented  itself  as  a  method  of  understanding  the 
development  and  adoption  of  expert  systems  applications.  This  was  opportune  because  it 
recommended  the  case  study  method  as  an  appropriate  method  for  analyzing  the 
implementation  of  an  expert  system.  Although  the  information  systems  and  computing 
sciences  literature  have  historically  shown  a  bias  toward  quantitative  methods  of  inquiry, 
this  study  asserted  a  preference  for  the  use  of  the  qualitative  methods  (Eliot  1992).  The 
author  believed  the  case  study  method  would  help  ground  and  augment  the  traditional 
empirical  laboratory-setting  examinations  (Eliot  1992). 

Lastly,  the  use  of  a  case  research  strategy  in  the  studies  of  information  systems 
was  examined.  Case  research  strategy,  which  uses  qualitative  techniques,  is  more 
appropriate  than  the  quantitative  techniques  that  the  author  cites  as  having  caused 
dissatisfaction  because  of  their  complexity  and  restrictions  (Benbasat  19871.  The  author 
then  presents  three  reasons  to  use  case  research  strategy  that  are  applicable;  the 
researcher  can  study  IS  in  a  natural  setting,  the  researcher  can  answer  the  questions  that 
lead  to  understanding  the  nature  and  complexity  of  the  processes  taking  place,  and  it  is  an 
appropriate  way  to  research  a  previously  little-studied  area  (Benbasat  1987V 
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METHODS  AND  PROCEDURES 

Who  was  Involved  in  the  Study? 

Although  the  transition  to  the  new  system  affected  everyone  in  the  Health  Plan, 
there  were  only  a  few  persons  directly  involved  with  the  implementation  project.  I,  of 
course,  had  an  integral  role  as  both  a  participant  observer  and  evaluator.  This  role  was 
difficult  and  frustrating  at  times,  but  it  was  not  without  its  rewards.  When  it  came  to  the 
project,  I  had  the  luxury  of  broad  decision  discretion,  macro-management,  and 
considerable  input  into  the  overall  process. 

I  also  had  the  opportunity  to  work  with  many  people,  with  mostly  pleasurable 
experiences.  The  ones  directly  involved  with  the  implementation  were  the  Director  for 
Administration,  the  manager  for  administrative  support,  the  purchasing  specialist,  the 
telecommunication  specialist,  the  software  vendor,  and  representatives  from  the 
Management  Information  Systems  (MIS),  marketing,  and  finance  departments. 

As  for  the  people  involved  with  the  evaluation  of  the  system,  a  randomized  typical 
case  sample  was  drawn  from  a  representative  pool  of  volunteers  from  the  health  plan. 

The  selection  of  this  representative  pool  was  done  with  the  cooperation  of  key  informants 
and  the  purchasing  specialist.  This  was  done  to  demonstrate  to  those  unfamiliar  with  the 
process  what  a  typical  experience  was  like,  and  not  for  generalizations  of  the  sample  (in  a 
critical  instance  study  broad  generalization  are  of  little  or  no  value)  (Tatton  1987k 
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How  was  the  Study  Completed? 


The  Case  Study  Method 

There  were  many  methods  of  study  that  could  have  helped  achieve  the  goals  and 
objectives  of  the  project.  However,  based  on  the  limits  of  the  specific  situation  I  believed 
that  there  were  only  a  few  methods  that  were  a  good  fit.  First,  I  believed  that  this 
situation  warranted  the  attention  of  a  case  study.  The  definition  that  was  found  to  be  most 
helpful  in  this  determination  came  from  the  Government  Accounting  Office’s  (GAO) 
Case  Study  methodology  guide:  “A  case  study  is  a  method  for  learning  about  a  complex 
instance,  based  on  a  comprehensive  understanding  of  that  instance  obtained  by  extensive 
description  and  analysis  of  that  instance  taken  as  a  whole  and  in  its  context  (U.S.  General 
Accounting  Office  19901.” 

There  are  three  portions  of  the  definition  that  were  extremely  relevant  to  the 
situation  and  strengthen  the  argument  that  this  was  the  best  method.  First,  this  study 
epitomized  a  complex  instance.  It  was  very  elaborate  and  complicated,  it  had  multiple 
convoluted  issues,  and  it  involved  many  departments  and  activities.  Secondly,  the 
detailed  understanding  gained  over  the  past  eight  months,  which  was  based  on 
comprehensive  understanding,  was  obtained  by  extensive  description  and  analysis.  And 
lastly,  since  this  study  involved  so  many  unquantifiable  aspects  of  the  health  plan,  it  was 
best  presented  holistically,  and  in  context.  This  project,  and  the  environment  that  it 
operated  in,  lent  itself  perfectly  to  the  case  study  method. 
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A  Critical  Instance 


The  next  aspect  in  the  study  development  process  was  the  determination  of  the  case 
study  type.  There  are  six  types  of  applications  for  case  study  methods;  illustrative, 
exploratory,  critical  instance,  implementation,  program  effects,  and  cumulative  (U.S. 
General  Accounting  Office  19901  Each  method  has  particular  strengths,  weaknesses, 
advantages,  and  disadvantages  for  various  situations  (with  each  situation  having  some 
design  decisions  that  are  more  relevant  than  others).  Taking  the  pertinent  information 
into  account,  and  after  careful  examination  of  the  design  decisions  of  each  method,  the 
literature  suggested  that  the  critical  instance  application  method  was  the  most  appropriate 
fU.S.  General  Accounting  Office  1990.  The  following  table  (Table- 1)  summarizes  the 
differences  between  the  varying  methods  and  illustrates  this  choice. 


I,  ■  -  ..  .  .  .  .; ...  .  ■  ■ 

..... ..... 'X>: .. ,  .. 

iv&i;: . 1  ...l.  itiS 

Basis  for  site  selection 

If  multi-method 
Pre-structuring 

Types  of  Data 

| Sequence  of  analysis 

Reporting 

Typical, 

representative, 

cluster 

Concurrent 

Low,  moderate 
Qualitative 
only,  qualitative 
and  quantitative 

Within  sites 

Narrative 

Convenience, 
unique  interest 

Concurrent 

Low,  moderate 
Qualitative  only, 
qualitative  and 
quantitative 
Within  sites 

Narrative 

Best-worst  case,  bracketing, 
typical,  representative, 
cluster,  probability 

Before,  concurrent,  after 
Moderate,  high 

Qualitative  only,  qualitative 
and  quantitative 

Within  sites,  then  across 
sites 

Thematic 

Table-1 
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A  key  point  of  the  critical  instance  case  study  application  that  was  particularly 

relevant  is  succinctly  presented  in  the  following  quotation. 

“The  critical  instance  case  study  examines  one,  or  very  few,  sites  for  one  of  two 
purposes  (for  our  situation,  only  the  first  purpose  is  relevant).  First,  a  very  frequent 
application  is  the  examination  of  a  situation  of  unique  interest,  such  as  the  Three  Mile 
Island,  or  the  Challenger  disaster.  There  is  little  or  no  interest  in  generalizability.  The 
instance  is  not  “selected”  by  us;  rather,  we  are  called  to  it  (U.S.  General  Accounting 
Office  19901” 

Which  was  exactly  what  had  occurred  in  this  instance. 

Qualitative  Descriptions 

This  case  study  needs  to  describe,  in  great  detail,  two  aspects  of  the  process.  The 
first  was  the  implementation  of  the  automated  system.  This  was  done  in  depth,  in  detail, 
in  context,  and  holistically  (Patton  1987V  This  lent  itself  to  the  qualitative  method  with 
direct  observations  and  “thick”  descriptions,  which  emphasized  the  importance  of  getting 
close  to  the  people  and  the  situations  being  studied  in  order  to  understand  personally  the 

i 

realities  and  minutiae  of  the  project  lifecycle  (U.S.  General  Accounting  Office  19901. 
Since  there  were  no  quantifiable  aspects  to  this  part  of  the  process,  and  this  study  aims  at 
individualized  outcomes,  the  qualitative  case  method  was  even  more  appropriate  (Patton 
19871.  The  second  aspect  of  the  process,  which  dealt  with  the  evaluation  of  the 
implementation,  is  presented  in  detail  in  the  next  section. 

The  Qualitative  Evaluation 

The  only  truly  appropriate  method  for  the  evaluation  of  this  system  implementation 
was  the  qualitative  method.  It  accommodated  this  study’s  need  for  flexibility  and 
captured  the  essence  of  the  experience.  But  since  this  study  could  have  been  done  either 
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quantitatively  or  qualitatively,  and  since  each  has  advantages  and  disadvantages, 
information  will  be  presented  to  explain  this  decision. 

First,  a  major  tenet  of  quantitative  analysis  is  its  stringent  methods  and  structures 
for  data  collection.  A  hypothesis  is  first  framed,  then  data  is  collected  to  either  support  or 
refute  the  claim  (Xing  1987V  This  would  not  have  been  appropriate  for  this  situation. 

The  project  was  started  with  no  preconceived  notions,  there  was  no  hypothesis  to  test. 

The  first  four  months  were  spent  learning  the  process  and  only  after  that  were  we  in  the 
position  to  develop  the  evaluation.  This  situation  and  the  type  of  approach  suggested  the 
qualitative  evaluation  and  reinforces  said  argument. 

The  next  point,  which  also  bolstered  the  appropriateness  of  a  qualitative  evaluation, 
lied  in  the  fact  that  there  was  no  written  plan  or  set  of  defined  tasks  (King  1987T  Since 
trying  to  retrospectively  construct  one  from  historical  information  was  impossible,  the 
only  method  still  justifiable  was  the  qualitative  method  of  observation  (King_1987). 
Lastly,  as  the  literature  review  illustrated,  this  was  a  unique  case.  Since  there  were  no 
common  features,  it  demanded  qualitative  inquiry  to  catch  the  true  flavor  of  the  program, 
and  to  discover  the  particular  set  of  elements  that  made  it  work  (Van  Maanen  1983L 
Measuring  the  Effectiveness 

Using  a  qualitative  evaluation  to  measure  the  effectiveness  of  the  implementation 
presented  many  interesting  choices  in  the  evaluation  instrument  design.  After  careful 
research  the  following  evaluation  design  was  utilized:  A  qualitative  analysis  with  the 
combination  of  open-ended  interviews  of  a  randomized  typical  case  sample  of  people  in 
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different  status  positions,  and  direct  participant  observations  (because  this  design  satisfies 
two  of  the  four  basic  tenets  of  Denzin’s  theory  on  data  collection  it  adequately 
compensates  for  any  problems  of  rival  causal  factors)  (Patton  1987V  Now  for  the 
questionnaire  itself. 

Open  Ended  Interview  Questions 

The  standardized  open-ended  interview  consisted  of  a  set  of  questions  carefully 
worded  and  arranged  for  the  purpose  of  taking  each  respondent  throughout  the  same 
sequence  and  asking  each  respondent  the  questions  with  essentially  the  same  words 
(Patton  1987k  This  method,  which  was  very  appropriate  for  the  situation,  is  traditionally 
used  when  there  is  limited  time  for  the  interview,  the  evaluation  is  gathering  opinion 
before  and  after  and  event,  and  it  is  important  to  minimize  the  variation  in  the  questions 
posed  to  the  interviewees  (Patton  19871. 

With  this  method  in  mind,  there  are  basically  six  kinds  of  questions  that  can  be 

asked  of  people  (Patton  1987).  For  the  purposes  of  the  evaluation,  only  the  three  types 

that  were  relevant  to  the  situation  were  used  in  the  creation  of  the  questionnaire. 

Knowledge  questions  are  aimed  at  finding  what  factual  information  the 
respondent  has.  The  assumption  here  is  that  certain  things  are  considered  to  be 
known.  These  things  are  not  opinions,  they  are  not  feelings,  but  rather  they  are 
things  that  one  knows.  They  are  the  facts  of  the  case  (Patton  1987). 

Opinion/Belief  questions  are  aimed  at  understanding  the  cognitive  and 
interpretive  processes  of  people.  Answers  to  these  questions  tell  us  what  people 
think  about  the  experience,  or  about  a  specific  process  (Patton  19871. 

Background/Demographic  questions  concern  the  identifying  characteristics  of 
the  person  being  interviewed.  They  are  distinguishable  from  the  knowledge 
questions  primarily  by  their  routine  nature  (Patton  19871. 
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With  the  above  fore  mentioned  concepts  in  mind,  a  standardized  open-ended 
interview  questionnaire  was  created.  It  included  questions  that  represented  each  of  the 
question  types  identified  in  the  previous  section  (these  questions  were  created  using  well- 
established  guidelines  for  content,  format  and  order)  (Patton  1987V  It,  as  well  as  a 
background  and  guideline  sheet  is  attached  as  Appendix-A  for  your  convenience. 


What  About  Validity  and  Reliability? 

Case  study  methods  can  use  two  tactics  for  achieving  measurement  validity: 
multiple  sources  of  evidence  and  using  the  chain-of-evidence  technique  in  data  reduction. 
Because  the  first  technique  lends  itself  to  my  study,  I  will  be  using  multiple  data  sources 
exclusively.  This  method  will  require  "thick"  description  in  order  to  get  enough 
information  to  check  for  trends,  to  rule  out  competing  explanations,  and  to  corroborate 
findings  (U.S.  General  Accounting  Office  1990k  Eight  techniques  can  be  used  to  collect 
information,  but  for  the  purposes  of  this  study,  I  will  be  concentrating  on  two  of  the 
methods  (open-ended  interviews  and  direct  observations)  (Yin  19941  (U.S.  General 
Accounting  Office  1990k 

As  for  reliability,  the  extent  to  which  a  measuring  procedure  can  produce  the  same 
results  on  repeated  trials,  there  are  two  relevant  points  for  the  content  analysis  portion  of 
this  study  (U.S.  General  Accounting  Office  1989k  First,  since  many  of  the  problems 
associated  with  content  analysis  are  due  to  inter-rater  reliability,  and  this  study  employed 
only  one  rater,  this  aspect  of  rater  reliability  is  not  applicable  (U.S.  General  Accounting 
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Office  1989T  Secondly,  pretesting  of  the  coded  categories  resulted  in  an  97  percent 
agreement  factor.  This  factor,  which  can  be  used  as  a  gauge  for  the  reliability  of  content 
analysis,  is  well  above  acceptable  levels  (80-90)  and  speaks  highly  of  the  coded 
categories  and  the  rating  reliability  (TJ.S.  General  Accounting  Office  1989s).  These  two 
points  combined  provide  reasonable  assurances  for  reliability  and  create  sufficient 
conditions  for  the  validity  of  the  study. 
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DISCUSSIONS 


Organizational  and  Project  Background 
This  portion  of  the  paper  will  revolve  around  the  objectives  identified  earlier. 

This  will  be  the  basis  of  the  project.  But  before  proceeding,  a  little  background  is 

warranted.  The  health  plan,  as  it  exists  today,  is  a 
mixed  model  HMO  (network  and  group)  that  operates 
in  the  Washington  metropolitan  area  (over  18 
counties  and  the  city  of  Baltimore).  It  has  over  90,000 
members  and  is  affiliated  with  over  forty  hospitals.  It 
provides  care  through  almost  3,500  credentialed 
physicians  at  six  satellite  clinics,  single  and  multi¬ 
specialty  group  offices,  and  private  practices.  It  is 

Figure  1 

currently  NCQA  accredited  and  consistently  ranks  high  in  regional  satisfaction  surveys. 

The  last  few  years  have  been  rocky  for  the  organization.  The  marketplace  has 
become  very  competitive  and  the  organization  has  not  fared  well.  It  has  been 
unprofitable  for  the  last  two  plus  years  and  the  University  (the  parent  company  if  you 
will)  has  been  reevaluating  the  desirability  of  ownership/management  of  this 
organization.  The  looming  possibility  of  acquisition  or  merger  has  filled  the  organization 
with  feelings  of  apprehension  and  uncertainty.  For  many  employees,  this  is  not  a 
desirable  place  to  work  (exemplified  by  the  constant  employee  retention  problems). 
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From  my  Perspective 

When  I  arrived  at  the  health  plan  I  had  certain  perceptions  (more  appropriately 
misconceptions)  about  what  it  would  be  like  to  work  in  this  civilian  organization.  Based 
on  brief  conversations  and  the  introductory  tour  there  was  an  expectation  of  a  state-of- 
the-art  facility  (both  in  operation,  and  resources).  In  some  respects  this  was  the  case,  in 
others,  this  was  sorely  mistaken.  There  were,  and  still  are,  many  aspects  of  the 
organization  that  are  clearly  deficient.  The  state  of  the  health  care  environment  in 
general,  and  the  local  marketplace  have  surely  caused  many  of  the  current  problems  and 
crisis’s.  But  they  are  not  the  only  attributable  causes.  Over  the  last  twenty-five  years  it 
appears  that  the  organization  has  outgrown  its  expertise  (past  poor  decisions,  antiquated 
processes,  and  lack  of  capital  improvements)  and  has  found  itself  in  the  position  of  being 
uncompetitive.  With  this  background  information  in  mind,  the  next,  and  remaining  sub¬ 
sections  will  be  devoted  to  the  project  itself. 

Project  Management 

To  complete  the  automation  project,  five  main  tasks  were  broken  into  multiple 
sub-projects.  Appendix-B  show  the  various  tasks,  sub-projects,  duration,  and  completion 
status  of  the  objectives  of  the  project  (Gantt  chart).  This  chart,  which  was  a  living 
document,  was  constantly  being  revised  during  the  project  lifecycle  to  reflect  and  adjust 
for  the  many  deliverable  slips.  The  tasks  and  sub-projects  identified  loosely  follow  the 
objectives  presented  earlier  and  are  the  outline  for  the  next  sections. 
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The  First  Objective  -  Analyze  and  learn  the  Plan’s  Purchase  Order  System 

This  objective  was  difficult  and  time  consuming.  Not  only  because  of  the 
intricacies  of  the  objective  itself,  but  also  because  it  was  my  first  experience  with  the 
people  involved.  Initially,  time  was  spent  in  briefings  surveying  background  papers  and 
information  on  the  plan's  purchasing  process.  Considerable  time  was  also  spent 
observing  the  processes  in  action  and  walking  through  the  process  step  by  step.  Meetings 
were  held  on  numerous  occasions  and  rudimentary  documents  were  created.  This 
constituted  the  preliminary  learning  phase  of  the  project.  This  was  done  to  determine  the 
feasibility  of  the  project.  At  the  conclusion  of  this  phase,  the  decision  was  made  to 
continue  with  the  project.  The  consensus  was  that  the  implementation  of  the  project 
would  result  in  noticeable  increases  in  purchasing  functionality  and  productivity. 

Continuing  with  the  learning  phase  of  the  project,  we  next  identified  the 
stakeholders  and  goals  of  the  project  (this  was  done  through  a  chain  of  event's  technique 
and  direct  observations).  Significant  time  was  spent  with  the  major  stakeholders  in  an 
effort  to  ensure  buy-in.  We  were  determined  to  identify  and  institute  adequate  “special” 
procedures  to  account  for  the  numerous  special  circumstances  that  invariably  would 
occur.  Although  the  draft-review-revision  cycles  were  lengthy  and  frustrating,  in  the  end, 
they  became  worthwhile,  because  buy-in  was  achieved  from  all  parties. 
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For  the  process  itself,  the  actual  step-by-step  procedures  of  purchasing.  There  was, 
as  stated  earlier,  no  documentation,  no  policy,  or  procedures  in  place.  Everything  was 
done  by  a  "rule  of  thumb,"  and  "that's  the  way  we've  always  done  it"  mentality.  This  type 
of  “system”  by  its  very  nature,  is  unreliable  and  inconsistent.  The  ironic  aspect  of  this 
situation  was  that,  in  concept,  it  could  have  easily  been  corrected.  The  actions  and 
outcomes  in  this  process  were  predictable.  Since  there  was  predictability  in  the  process, 
the  actions  and  outcomes  could  be  linked  together  to  create  a  map  of  the  entire  process. 
This  was  done  and  is  enclosed  as  Appendix-C  (refinement  of  these  “process  maps” 
continued  throughout  the  project).  These  maps,  which  were  well  received  and  accepted, 
were  the  basis  of  the  policies  and  procedures  that  were  later  developed. 

This  first  objective,  learning  the  plan’s  purchase  order  system,  resulted  in  the 
identification  of  the  stakeholders  of  the  process  and  the  creation  of  the  goals  of  the 
project  and  process  maps.  The  combination  of  this  information  and  understanding 
became  the  foundation  for  the  project.  The  rest  of  the  objectives  were  built  upon  the 
information  presented  in  this  section. 

The  Second  Objective  -  Evaluate  and  Select  a  Software  Vendor 
This  started  with  an  investigation  of  the  feasibility  and  cost/benefit  of  purchase 
order  software.  After  contacting  various  vendors  and  peers  at  other  organizations,  to 
determine  costs  and  solicit  recommendations  for  purchase  order  tracking  software,  we 
learned  that  many  like  organizations  either  had  no  software  application  in  place,  or  had  a 
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“home  grown”  proprietary  application.  After  an  extensive  search  we  were  able  to 
identify  three  vendors  that  offered  possible  solutions.  Fact  sheets  from  the  vendors  about 
the  three  software  products  are  included  in  Appendix-D. 

Demonstrations  of  the  software  product  were  coordinated  and  demo- versions  of 
the  packages  were  obtained.  Working  with  the  vendors,  learning  the  software,  and 
identifying  the  strengths  and  weakness  of  the  products  was  cut  and  dry.  At  the  time  of 
the  decision,  and  after  careful  evaluation,  there  was  only  one  product  that  had  the 
capabilities  that  we  were  looking  for.  The  decision  was  made  to  choose  this  product,  it 
was  called  the  SMARRT  system  (Supply  Management  Automated  Reporting 
Replenishment  and  Tracking  produced  by  BT  Office  Products  International). 

The  Third  Objective  -  Coordinate  &  Resolve  Implementation  Issues  with  Vendor 
Getting  the  software  product  installed  and  running  was  the  next  course  of  action. 
This  involved  conversations  and  meetings  with  the  vendor’s  support  staff  and  the  health 
plan’s  MIS  department.  Time  tables  and  system  change  requests  were  created,  and  after 
the  initial  preparation,  the  application  was  installed  and  up  and  running  in  less  than  a 
week.  This  was  important  because  SMARRT  application  training  was  a  lengthy  part  of 
the  project  and  could  not  be  started  until  this  step  was  completed. 

The  next  requirement  was  the  submission  of  the  health  plan’s  accounting  cost 
centers  to  the  vendor  for  incorporation  into  the  final  database.  This  information  needed 
to  be  provided  and  approved  by  the  finance  directorate.  In  concept,  this  task  was  simple, 
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but  since  the  finance  directorate  was  undergoing  sweeping  changes,  it  proved  extremely 
difficult.  Until  October  1996,  the  organization  looked  at  the  annual  budgeting  process 
from  the  macro  perspective.  Often  not  budgeting  down  to  the  department  level  for  many 
account  types.  But  after  the  reorganization  of  March  1996  (a  reorganization  in  which  the 
COO,  CFO,  and  Medical  Director  were  removed),  the  new  front  office  looked  at  issues 
from  a  different  perspective.  One  of  these  issues  was  the  annual  budget  process. 

In  October  1996,  the  (new)  CFO  introduced  and  explained  to  the  health  plan  the 
new  annual  budget  process.  This  process  utilized  zero-based  budgeting  and  a  detailed 
departmental  allocation  matrix.  This  was  not  well  received  (even  though  it  was  clearly 
appropriate  given  the  situation)  and  a  new  and  difficult  process  for  some  managers  to 
comply  with.  It  was  at  this  time  that  we  were  trying  to  provide  the  cost  center 
information  to  the  vendor  for  the  database  incorporation.  Needless  to  say,  we  were  not 
finance’s  highest  priority.  The  original  hopes  of  deploying  the  new  purchasing  system  on 
the  first  of  January  1997  were  quickly  dashed  when  finance  could  not  arrive  at  a 
consensus  concerning  the  internal  cost  centers  and  the  external  satellites.  This  condition 
dragged  on  until  the  beginning  of  December  1996  before  it  was  finally  resolved  (final 
cost  centers  are  included  as  Appendix-El).  This  resulted  in  the  first  major  delay  of  the 
project,  the  deployment  schedules  slipped  by  more  than  a  month. 

The  last  issue  to  be  addressed  for  this  objective  concerned  the  management  of  the 
SMARRT  system.  From  a  system  administration  perspective,  there  were  a  number  of 
issues  that  needed  to  be  resolved  and  addressed.  These  issues  involved  working  with  the 
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vendor  identifying  and  resolving  all  application  management  issues  (back-up,  filters,  log¬ 
on  procedures,  users  database,  and  upgrade  cycles.  The  identification  and  resolution  of 
these  issues  went  well  and  did  not  present  any  problems  to  the  long-term  viability  of  the 
project  (some  of  this  documentation  is  included  as  Appendix-E2). 

The  Fourth  Objective  -  Develop  and  Implement  the  Training  Plan 

Shortly  after  the  SMARRT  system  was  installed  we  began  application  training. 
This  training  took  longer  than  expected,  but  considering  the  issues  involved,  went 
reasonably  well.  The  users  of  the  system  had  no  practical  experience  on  a  computer  let 
alone  using  a  Windows  based  tracking  system.  Even  though  this  was  known  prior  to  the 
start  of  the  training,  its  significance  was  not  realized  until  the  second  week  of  training. 

During  the  second  week,  we  realized  the  training  timelines  were  too  aggressive 
and  needed  to  be  adjusted.  The  users  of  the  system  could  not  adequately  retain  the 
application  information  and  were  having  difficulty  comprehending  some  of  the  concepts. 
The  training  plan  was  extended  considerably.  The  original  plan  called  for  three  months 
of  training,  this  was  extended  to  seven  months.  This  had  an  impact  on  the  deployment  of 
the  project,  but  it  was  not  as  dramatic  as  one  would  think.  By  adjusting  the  order  of  the 
topics  covered,  and  concentrating  on  the  “operational”  subjects  first,  we  were  able  to 
create  a  “survival  list”  of  concepts.  This  list  addressed  the  concepts  and  subjects  that 
were  most  important  to  the  immediate  operation  of  the  system.  By  training  on  these 
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subjects  first,  we  were  able  to  deploy  the  system  only  a  month  later  than  expected.  This 
was  the  second  major  slip  for  the  project  (the  first  one  being  the  cost  center  predicament). 

The  actual  training  sessions  consisted  of  one  hour,  one-on-one  sessions,  up  to 
three  times  a  week.  These  sessions  were  held  in  the  users’  workspace,  on  the  users’ 
computer,  working  around  the  users’  schedule.  This  was  done  to  minimize  the  disruption 
and  increase  the  level  of  comfort  associated  with  these  sessions.  The  users’  were  given 
clearly  defined  objectives  for  the  session  and  short  exercises  that  were  targeted  at 
retention  and  conceptual  comprehension.  The  training  plan  was  broken  down  into  four 
main  subjects;  Windows  survival  skills,  working  with  purchase  orders,  working  with 
inquiries  and  reports,  and  maintaining  the  databases.  The  first  two  subjects  were 
considered  critical  to  the  operation  of  the  system  and  were  addressed  first  (as  part  of  the 
“survival  list”  presented  earlier).  The  other  two  subjects  were  addressed  later  and  did  not 
impact  on  the  day-to-day  operation  of  the  application. 

The  individual  session  training  plan’s  were  created  prospectively  and  adjusted  and 
refined  concurrently.  Every  session  had  a  “draft”  lesson  plan  that  was  used  to  guide  the 
session.  But  as  each  session  progressed,  the  lesson  plan  was  adjusted  to  reflect  the 
personal  habits  and  mannerisms  of  the  users.  These  adjustments  were  included  into  the 
revised  lesson  plans  and  returned  to  the  users  for  comment  and  possible  modification. 
This  resulted  in  a  customized  training  manual  that  was  used  for  and  created  during  the 
training  sessions  (included  as  Enclosure  1).  This  substantial  effort  produced  a  manual 
that  was  both  complete  and  user-friendly.  This  site-specific  training  manual 
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supplemented  the  site-generic  user  manual  provide  by  the  vendor.  The  combination  of 
these  two  manuals  provided  excellent  coverage,  content,  and  examples  of  the  information 
needed  for  the  proper  operation  of  the  application. 

The  Fifth  Objective  -  Develop  the  New  Policies  and  Procedures 

The  Proverbial  Can  of  Worms 

At  the  beginning  of  this  project,  the  intent  of  this  objective  was  narrow  and 
succinct;  the  creation  and  implementation  of  a  purchase  order  policy  for  the  organization 
and  internal  purchase  order  procedures  for  the  department.  Unfortunately,  this  hoped  for 
simplicity  would  not  be  the  case,  because  this  objective  quickly  became  more  involved 
and  complicated.  This  revealed  itself  in  many  ways,  but  the  main  difficulty  was  one  of 
annoying  diversions.  Before  we  could  address  the  primary  tasks,  we  learned  that  there 
were  other  related  health  plan  issues  that  were  deficient  or  had  never  been  addressed. 

This  put  us  in  the  position  of  having  to  accomplish  many  secondary  tasks  that  were  not 
included,  or  even  addressed  in  the  preliminary  project  discussions.  What  follows  are  the 
chain  of  events  that  outline  the  steps  taken  and  the  deliverables  created  as  we  worked  our 
way  towards  completion  of  the  primary  tasks. 

New  Purchase  Request  Forms 

The  old  process  for  requesting  goods  or  services  was  done  with  the  use  of  the 
health  plan  purchase  orders  (PO)  (included  as  Appendix-Fl).  These  forms  needed  to  be 
filled  out  and  “approved”,  submitted  to  the  purchasing  department  for  assignment  of  a 
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sequential  PO  number,  and  then  faxed  to  a  vendor.  These  presequenced  purchase  order 
forms  were  lying  around  in  every  nook  and  cranny  of  the  organization.  They  (the  forms 
that  were  lying  around)  were  the  exact  same  forms  that  were  faxed  to  the  vendors.  All 
that  was  needed  was  a  PO  number  and  a  signature  (any  signature). 

From  a  fiscal,  administrative  and  management  perspective,  there  was  a  level  of 
discomfort  with  this  type  of  control  mechanism  for  the  process  (or  should  we  say  lack  of). 
There  was  very  little  that  could  prevent  the  unauthorized  use  of  these  forms  for  improper 
obligations  to  the  organization.  Worst  case  scenario,  it  was  feared  that  this  lack  of 
control  would  create  an  opportunity  for  embezzlement,  best  case,  it  was  known  that  this 
process  facilitated  rampant  abuse  of  unapproved  authorized  purchases.  In  either  case,  it 
was  not  the  proper  way  to  manage  the  operation. 

It  became  necessary  to  redesign  the  process  a  little.  One  of  our  first  tasks  was  the 
creation  of  new  Purchase  Request  Forms  (PRF)  (included  as  Appendix-F2).  These 
electronic  forms  were  designed  to  replace  the  purchase  orders  that  were  currently  being 
used.  They  would  be  widely  distributed  for  use  within  the  organization,  and  could  be 
completed  and  printed  electronically.  The  use  of  these  new,  more  functional  and 
comprehensive  forms  would  effectively  remove  the  purchase  order  form  from  the  hand  of 
the  health  plan  employee  and  increase  the  effectiveness  of  the  process.  New  purchase 
orders  (included  as  Appendix-F3)  would  now  only  be  able  to  be  created  on  the  SMARRT 
system  (access  to  this  system  is  restricted)  thus  effectively  eliminating  the  risk  associated 
with  unlimited  access  to  purchase  order  forms. 
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The  next  task  to  be  completed  was  the  introduction  of  the  new  forms  and 
destruction  of  the  old  forms.  This  was  done  over  time  to  minimize  any  possible 
disruption  to  the  employees  of  the  health  plan.  Instructions,  definitions,  and  samples  to 
be  used  in  conjunction  with  the  new  forms  were  created  (included  as  Appendix-F4)  and 
distributed  throughout  the  organization  by  way  of  memos,  the  weekly  newsletter,  and 
Email.  On  March  3, 1997  the  organization  began  using  the  new  forms  and  due,  in  main 
part  to  the  effective  preparation,  the  transition  went  very  well  (the  forms  have  been  in  use 
since  the  introduction  with  no  associated  problems).  The  acceptance  and  use  of  these 
new  forms  was  a  major  accomplishment.  But  it  was  only  one  part  of  this  objective. 
Health  Plan  Signature  Authority  Cards  and  Policy 

The  next  aspect  of  the  purchasing  department’s  new  procedures  that  was 
considered  integral  was  the  prospective  approval  of  all  health  plan  purchases.  In  the  past 
this  was  mostly  done  retrospectively,  and  sometimes  not  at  all.  This  was  inappropriate 
and  in  need  of  a  change.  To  resolve  this  matter  we  wrote  into  the  purchasing  department 
procedures  the  requirement  to  validate  the  purchase  requests  as  they  were  submitted. 
Guidelines  for  this  review  process  were  created  and  were  well  received. 

There  were  only  a  few  items  on  the  purchase  request  that  need  to  be  validated. 
One  such  item  was  the  signature  block  (for  proper  authorization  for  the  purchase).  This 
would  traditionally  be  accomplished  through  a  visual  signature  comparison  check  that 
would  commonly  be  done  by  comparing  the  signature  on  the  request  with  the  signature 
on  the  accounting  signature  cards.  To  my  dismay,  when  I  inquired  about  the  signature 
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cards,  I  was  informed  that  “we  used  to  use  them,  but  we  haven’t  for  the  last  couple  of 
years”.  The  accounts  payable  department  “recognized”  most  of  the  signatures  from 
memory  and  called  the  requester  if  there  was  a  “problem."  This  was  all  done 
retrospectively,  after  the  health  plan  had  already  been  billed  for  the  purchase. 

Again,  this  was  inappropriate  and  inadequate.  With  the  health  plan’s  permission, 
we  reintroduced  a  revised  signature  card  to  the  managers  with  signature  authority 
(included  as  Appendix-F5).  There  was  some  confusion  at  first,  because  no  one  really 
knew  which  managers  had  authority  for  what  amounts.  After  conversations  with  the 
CEO  and  the  CFO  we  were  able  to  straighten  this  out  and  complete  this  process 
(signature  authority  list  included  as  Appendix-F6).  During  the  completion  of  this  process 
we  learned  about  the  next  issue  that  needed  to  be  resolved.  Many  of  the  managers  who 
now  had  signature  authority,  did  not  know  what  that  meant.  There  was  no  organizational 
policy  that  detailed  and  explained  the  signature  authority  process.  This  was  our  next  task. 

After  reviewing  the  old  (very  old)  documentation,  and  consulting  with  the  CEO, 
CFO,  Director  for  Administration,  and  the  University,  we  were  able  to  create  and 
implement  the  new  signature  authority  policy.  This  policy  delineated  the  differences 
between  capital  and  non-capital  items  as  well  as  defining  the  different  levels  of 
authorization  (included  as  Appendix-F7).  This  policy  was  sorely  needed  and  was  well 
received.  With  the  signature  authority  cards  and  new  policy  in  place  we  were  now  in  a 
position  to  address  the  primary  issue. 
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The  Purchasing  Policy 

This  policy,  which  was  part  of  the  primary  goal  of  this  objective,  was  also  the 
most  lengthy  to  completion.  The  approval  process  for  this  policy  seemed  to  drag  on 
endlessly.  This  was  due  to  circumstances  more  than  anything  else,  because  it  was  during 
this  time  that  another  wave  of  transitions  surged  through  the  organization.  As  mentioned 
earlier,  employee  retention  in  the  organization  had  become  a  significant  issue.  We  were 
loosing  a  handful  of  highly  skilled  employees  each  month  with  little  hope  or  success  in 
replacing  them.  For  this  project  alone,  this  period  saw  a  quite  a  few  changes.  The 
Director  of  Administration  left  the  health  plan,  quickly  followed  by  his  two  successors. 
This  period  of  transition  did  not  help  the  productivity  or  efficiency  of  this  project.  Each 
new  manager  required  the  explanation  and  re-explanation  of  the  same  topics  and  each 
wanted  to  ensure  that  their  perspectives  were  adequately  represented.  Although  this  was 
understandable,  it  was  also  time-consuming  and  somewhat  unproductive. 

After  countless  reviews  and  revisions  the  policy  was  finally  approved  and 
distributed.  The  final  policy  that  went  out  was  significantly  different  from  the  original 
(included  as  Appendix-F8).  Although  it  was  still  mainly  based  on  the  “process  maps”,  it 
had  been  dramatically  reduced  in  size  and  was  greatly  simplified.  This  was  done  at  the 
request  of  the  Executive  Management  Team  in  an  effort  to  ease  its  introduction  to  the 
organization.  Introduction  of  the  policy  to  the  organization’s  managers  was  the  next  task 
to  be  completed.  This  was  done  to  facilitate  buy-in  and  answer  any  questions  they  might 
have  had.  Announcements  for  the  overview  sessions  were  made  through  the  normal 
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channels  and  the  kick-off  presentations  were  held  over  a  two  week  period  (included  as 
Appendix-F9).  These  presentations  went  well  and  resulted  in  greater  acceptance  and 
compliance  with  the  new  policy. 

The  Purchasing  Department’s  Procedures 

The  last  task  to  be  completed  as  part  of  this  objective  was  the  internal  procedures 
for  the  purchasing  department.  This  task  was  simple  in  concept,  but  was  and  still  is 
difficult  in  practice.  Since  the  department  has  a  reputation  for  being  unreliable,  the  task 
that  was  addressed  centered  around  three  issues;  customer  service,  accountability  and 
creditability  (chosen  because  of  insights  gained  during  the  evaluation  of  the  old  system). 
These  tasks  were  completed  with  the  participation  and  active  involvement  of  the  Director 
of  Administration  and  the  members  of  the  purchasing  department. 

There  were  three  procedures  implemented  that  addressed  each  aspect  of  our 
perceived  deficiencies.  The  first  put  into  place  was  the  standardization  of  operations  and 
the  creation  of  performance  goals  (Appendix-FlO).  This,  in  effect,  was  our  promise  to 
our  customers.  It  was  the  measuring  stick  that  our  customers  could  use  to  rate  our 
credibility.  The  next  procedure  enacted  dealt  with  the  creation  of  a  departmental  activity 
schedule  (Appendix-Fl  1).  This  was  developed  to  create  an  environment  in  which  the 
department  could  become  accountable,  an  environment  that  would  allow  the  regular 
performance  of  certain  tasks  and  duties.  The  last  procedure,  which  was  developed  to 
address  our  problems  with  customer  service,  identified  the  need  to  provide  appropriate 
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customer  feedback  for  problem  shipments.  A  form  (Appendix-F12)  was  created  and 
appropriate  procedures  were  put  in  place  to  resolve  this  problem. 

These  procedures,  we  believed,  created  a  suitable  response  and  action  plan  for  the 
situation.  But  it  was  just  that,  a  plan.  As  with  all  plans,  effectiveness  is  most  often 
attributed  to  proper  implementation,  and  this  part  of  the  project,  unfortunately,  was  not 
implemented  correctly.  The  problems  that  occurred  could  generally  be  attributed  to  the 
lack  of  management  oversight.  As  stated  earlier,  there  was  a  period  in  which  this  project 
had  three  managers  in  quick  succession.  This  turnover  created  an  environment  in  which 
the  members  of  the  purchasing  department  felt  little  consequence  to  non-compliance  of 
policies  and  procedures  (although  these  changes  were  not  outwardly  resisted,  they  were 
not  embraced  either). 

When  it  came  time  to  change,  the  rule  of  thumb  appeared  to  be  “whatever  is 
easier”.  This  resulted  in  the  implementation  of  some  policies  and  not  others.  This 
situation  was  repeatedly  discussed  with  the  managers  of  the  operation  and  was  met  with 
understanding  and  agreement.  On  numerous  occasions  it  was  to  have  been  addressed, 
with  the  persons  involved,  but  to-date,  it  has  not  and  things  have  remained  the  same.  The 
department  continues  to  have  difficulties  with  its  levels  of  customer  service, 
accountability  and  creditability.  Although  this  portion  of  the  implementation  did  not 
work  as  desired,  it  is  expected  that  over  time,  and  with  the  active  involvement  of  the 
direct  manager,  this  situation  can  be  resolved. 
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The  Sixth  Objective  -  Opinion  and  Satisfaction  of  the  Old  System 

To  determine  the  opinion  and  satisfaction  of  the  old  system,  open  ended 
interviews  were  conducted  and  the  responses  were  studied  with  a  technique  called 
content  analysis.  Content  analysis,  which  can  be  used  for  examining  trends  over  time, 
was  used  because  of  its  ability  to  handle  information  that  is  difficult  to  combine  and 
analyze  (U.S.  General  Accounting  Office  1989).  This  type  of  analysis  is  generally 
defined  as  a  set  of  procedures  for  collecting  and  organizing  information  in  a  standard 
format  that  allows  analysts  to  make  inferences  about  the  characteristics  and  meaning  of 
written  and  other  recorded  material  (U.S.  General  Accounting  Office  1989T  This 
procedure  was  the  best  and  most  flexible  means  of  determining  the  opinion  and 
satisfaction  of  the  old  system.  To  explain  its  application,  the  following  background 
information  is  presented. 

There  are  five  steps  for  the  valid  utilization  of  content  analysis  that  must  be 
employed;  1)  determine  what  material  should  be  included,  2)  select  units  of  analysis,  3) 
develop  coding  categories,  4)  code  the  material,  5)  analyze  and  interpret  the  results  (U.S. 
General  Accounting  Office  1989k  Each  step  will  be  addressed  in  turn  and  its  relevant 
issues  will  be  presented.  For  step  one,  the  relevant  issue  is  sampling.  For  some 
applications  sampling  of  the  material  should  be  used  when  the  material  to  be  analyzed  is 
too  extensive.  For  this  application  however,  sampling  was  not  used  because  the  material 
to  be  analyzed  was  manageable.  For  step  two,  the  designation  of  the  units  of  analysis  is 
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the  important  issue.  For  this  study,  the  context  unit  will  be  the  open  ended  interview,  and 
the  recording  unit  will  be  the  groups  of  words  that  embody  the  discussion  of  the  issues. 

For  step  three,  the  coding  of  the  categories  is  the  pertinent  issue,  because  it 
provides  the  structure  for  grouping  the  recording  units  (XJ.S.  General  Accounting  Office 
1989V  There  are  three  standard  requirements  that  the  categories  must  meet;  they  should 
be  exhaustive,  they  should  be  mutually  exclusive,  and  they  should  be  independent. 
Categories  can  be  used  to  measure  three  qualification  levels;  space,  frequency,  and 
intensity  fU.S.  General  Accounting  Office  1989V  For  this  study,  we  were  interested  in 
the  quantification  of  frequency,  which  can  be  coded  by  tallying  the  number  of  times  each 
issue  or  statement  occurs  in  the  material  flJ.S.  General  Accounting  Office  1989V 
Directly  tied  to  this  are  two  assumptions  that  are  important  to  incorporate  when 
constructing  frequency  measures.  “First,  the  frequency  with  which  a  statement  occurs  is  a 
valid  indication  of  value  or  importance.  Second,  all  content  units  can  be  given  equal 
weight  and  therefore  each  one  can  be  compared  directly  with  every  other  (XJ.S.  General 
Accounting  Office  19891.” 

For  step  four,  the  relevant  issue  is  coding.  Material  can  be  coded  either  manually 
or  by  computer,  depending  on  the  resources  available  and  the  format  of  the  material  flJ.S. 
General  Accounting  Office  1989V  For  this  study  the  material  was  coded  manually.  For 
the  last  step,  step  five,  the  main  issue  is  analyzing  and  interpreting  the  results.  This  can 
be  done  many  ways,  none  of  which  are  unique  to  content  analysis.  For  this  study 
however,  the  coded  data  was  summarized  in  an  effort  to  discover  opinion  and  patterns. 
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Now  for  the  results,  Appendix-G  presents,  in  a  simple  format,  a  coding  and  tally 
sheet  that  measured  the  number  of  times  an  issue  was  addressed  during  the  pre¬ 
implementation  interviews.  This  sheet  is  organized  by  question  and  color  coded.  The 
questions  (discussed  earlier  in  the  open  ended  interview  question's  section)  and  the 
categories  are  broken  down  into  color  coded  positive  (green)  and  negative  (red) 
statements.  The  actual  statements  given  during  the  interviews  were  usually  different,  but 
they  all  revolved  around  only  a  few  select  themes.  Categories  were  developed  with  these 
“themes”  and  the  statements  were  re-coded  for  frequency  again. 

In  general,  the  pre-implementation  opinion  of  the  process  was  not  favorable. 
There  were  38  statements  made  that  would  generally  be  considered  unfavorable  or 
negative.  This  contrasts  with  the  3  statements  that  were  made  that  would  be  considered 
favorable  or  positive.  The  relative  frequency  of  these  statements  is  displayed  as  a  pie 
chart,  with  an  overwhelming  93  percent  negative  statement  rate.  This  clearly 
demonstrates  the  negative  opinion  of  the  pre-implementation  purchase  order  process  and 
provides  a  comparison  point  for  the  post-implementation  interview  analysis. 

The  subjects,  representing  a  wide  variety  of  positions  and  experiences,  were  fairly 
consistent  in  their  responses.  There  did  however,  appear  to  be  one  outlier.  This  outlier, 
which  made  generally  positive  statements  about  the  pre-implementation  process,  was  in 
stark  contrast  to  the  other  subjects.  This  subject  occupies  a  position  in  the  organization 
that  represents  an  influential  person  and  anecdotal  evidence  suggests  favoritism  for  this 
position  has  resulted  in  atypical  experiences  for  this  subject. 
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The  Seventh  Objective  -  Deployment  of  the  New  System 
This  project  used  a  phased  proto-type  implementation  with  deployment  of  the 
phases  occurring  over  the  four  months  of  February  through  May  1997.  This  type  of 
implementation  allowed  for  maximum  flexibility  and  concurrent  feedback.  There  were 
three  phases  of  the  deployment;  database  tracking,  implementation  of  new  forms  and 
associated  policy,  and  implementation  of  the  new  purchasing  policy  and  procedures. 

The  First  Phase 

The  first  phase,  which  started  in  the  second  week  of  February  1997,  consisted  of 
mainly  internal  purchasing  activities  that  were  transparent  to  the  rest  of  the  organization. 
It  started  with  portions  of  the  old  process  and  combined  them  with  portions  of  the  new 
process.  It  used  the  old  forms  and  policies  but  took  that  information  and  entered  it  into 
the  new  SMARRT  database  which  created  the  new  purchase  orders.  This  gave  the 
purchasing  department  more  time  to  become  familiar  with  the  operation  of  the  system 
and  the  creation  of  purchase  orders  (a  sort  of  practice  phase).  This  phase  lasted  about 
three  weeks  and  was  instrumental  in  instilling  confidence  in  the  users’  of  the  system. 

The  Second  Phase 

The  next  phase  of  the  deployment  was  the  institution  of  the  new  request  forms 
and  associated  policy.  This  began  in  March  and  lasted  the  entire  month.  During  this  time 
the  purchasing  department  accepted  both  types  of  forms.  However,  when  old  forms  were 
received  for  processing  we  took  the  opportunity  to  meet  with  the  originator  of  the  request. 
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This  was  done  so  that  we  could  introduce,  or  re-introduce  (whichever  the  case  may  have 
been),  the  new  request  forms.  We  also  took  the  time  to  explain  the  benefits  of  the  new 
form  and  answer  any  questions  they  might  have  had  (all  of  this  was  done  ad-hoc  and  in  a 
non  adversarial  manner).  The  new  forms  and  policy  were  received  well  and  by  the  end  of 
the  month  the  organization  had  entirely  switched  to  the  new  forms. 

The  Third  Phase 

The  last  phase  of  the  deployment  was  the  implementation  of  the  new  purchasing 
policy  and  the  internal  purchasing  procedures.  Since  the  organization  had  been 
previously  made  aware  of  the  upcoming  changes  on  numerous  occasions  and  via 
numerous  vehicles  the  actual  transition  was  anti-climactic.  It  had  been  expected  for  some 
time,  and  it  finally  occurred.  From  a  project  management  standpoint,  this  was  a  good 
thing.  It  meant  that  we  had  adequately  prepared  the  organization  for  the  change  and  it 
was  readily  accepted.  The  new  policy  worked,  and  continues  to  work,  well.  There  have 
been  little  or  no  associated  problems  with  the  policy.  On  the  other  hand,  as  discussed 
earlier,  the  internal  procedures  of  the  purchasing  department  have  continued  to  be 
inconsistent  at  best.  This  has  had  an  impact  on  the  “customer  satisfaction”  piece  of  the 
pie  and  will  be  discussed  further  in  the  next  sections. 
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Advantages  of  the  New  System 

The  advantages  of  the  system  are  many.  As  you  may  recall,  there  were  two  goals 
for  the  project;  increase  the  availability  of  useful  information,  and  improve  the  structure 
and  administrative  control  of  the  purchase  order  process.  To  a  major  extent,  we  have 
attained  these  goals.  The  attainment  of  these  goals  will  be  presented  and  discussed  from 
both  an  objective  and  subjective  perspective.  Some  aspects,  which  are  more 
appropriately  addressed  from  the  subjective  perspective,  will  be  presented  in  the 
evaluation  section.  The  other  aspects,  that  are  more  appropriately  addressed  from  the 
objective  perspective,  will  be  presented  in  this  section. 

These  aspects  deal  specifically  with  the  attainment  of  the  first  goal;  increasing  the 
availability  of  useful  information.  As  mentioned  in  the  introduction,  there  was  little  or  no 
useful  information  available  from  the  old  system.  It  was  very  antiquated.  The  new 
system,  on  the  other  hand,  has  many  advantages  over  the  old  system  and  has  already 
provided  a  wealth  of  information  in  its  short  existence.  This  section  will  present  some  of 
the  more  important  advantages  that  the  application  has  to  offer.  These  advantages  are 
clear-cut  without  objection  (hence  its  objective  presentation). 

The  new  system  offers  the  added  security  of  being  a  computer-based  application 
that  is  password  protected  and  restricted  from  general  network  access.  This  is 
advantageous  because  this,  and  only  this,  system  can  create  the  purchase  orders  for  the 
organization  (Appendix-Hl).  These  new  PO’s  differ  significantly  from  the  old  ones 
(there  should  be  no  confusion  in  the  accounting  department  or  from  the  vendors).  This  is 
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beneficial  from  both  an  administrative  and  fiscal  perspective  because  it  offers  reduced 
general  access,  secured  authorized  access,  and  clearly  differentiated  forms. 

The  next  advantage  comes  in  the  form  of  useful  information.  In  this  comparison, 
there  is  no  comparison.  The  old  system  offered  nothing  similar,  more  appropriately, 
nothing  at  all.  This  system,  since  it  is  tightly  integrated  with  a  database,  offers  a  wealth 
of  information  that  is  available  in  report,  queries,  and  graphical  formats  (see  table-2). 
This  information,  which  is  easily  accessible  through  the  application’s  point-and-click 
interface,  is  also  broken  down  by  cost  center,  month  and  fiscal  year.  The  database  can 
store  the  information  for  three  years  and  also  allows  exportation  of  the  data  into  a  myriad 
of  other  applications.  These  features  combined  create  a  system  that  is  clearly  superior  to 
its  predecessor  when  it  comes  to  the  amount  of  available  and  useful  information. 


SMARRT  Reports  and  Output 

Order  Detail  List  Report 

Appendix-H2 

Order  List  Report 

Appendix-H3 

Cost  Center  Usage  Report 

Appendix-H4 

Ship-To  Usage  Report 

Appendix-H5 

Order  List  by  Cost  Center  (exported  and  reformatted) 

Appendix-H6 

Purchase  Order  Tracking  (query  module) 

Integrated  Module 

Item  Usage  (query  module  and  report) 

Integrated  Module 

Top  n  Item  Usage  (by  volume/$)  (query  module  and  report) 

Integrated  Module 

Cost  Center  Analysis  Graphs  (graphing  module) 

Appendix-H7 

Ship-To  Analysis  Graphs  (graphing  module) 

Appendix-H8 

Purchase  Orders  by  Cost  Center  Pie  Charts  (exported) 

Appendix-H9 

table-2 
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The  Eighth  Objective  -  Evaluate  the  New  System 


To  determine  the  evaluation  of  the  implementation  of  the  new  system,  open  ended 
interviews  were  conducted  and  the  responses  were  studied  with  a  technique  called 
content  analysis.  The  application  of  this  technique  was  identical  to  the  procedures 
discussed  earlier.  This  was  done  to  allow  for  comparisons  of  the  pre  and  post  interviews 
and  a  determination  of  the  implementation’s  success  or  failure. 

Appendix-I  presents,  in  a  little  more  detailed  format  than  the  previous  version,  a 
coding  and  tally  sheet  that  measured  the  number  of  times  an  issue  was  addressed  during 
the  pre  and  post-implementation  interviews.  This  sheet  is  organized  by  question  and 
color  coded.  The  questions  and  the  categories  are  broken  down  into  color  coded  positive 
(green)  and  negative  (red)  statements.  The  actual  statements  given  during  the  interviews 
were  usually  different,  but  they  all  revolved  around  a  few  select  themes.  Categories  were 
developed  with  these  “themes”  and  the  statements  were  re-coded  for  frequency  again. 

In  general,  the  post-implementation  opinion  of  the  process  was  favorable.  There 
were  1 8  statements  made  that  would  generally  be  considered  unfavorable  or  negative  and 
34  statements  that  would  be  considered  favorable  or  positive.  This  is  significantly 
different  from  the  statements  obtained  in  the  pre-implementation  interviews  (which  were 
generally  unfavorable).  The  relative  frequency  of  these  statements  is  displayed  as  a  pie 
chart,  with  a  65  percent  positive  statement  rate.  This  compares  very  favorably  to  the  pie 
chart  that  represents  the  pre-implementation  opinion  of  the  process. 
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Not  only  does  the  post-implementation  pie  chart  represent  a  positive  opinion  of 
the  new  process,  more  importantly,  it  demonstrates  a  dramatic  shift  in  opinion  from  the 
subjects.  This  is  viewed  as  a  clear  indication  that  the  implementation  process,  by  and 
large,  was  successful.  More  careful  examination  of  the  coding  and  tally  sheet  reveals 
another  aspect  of  this  point  that  warrants  further  discussion.  This  aspect  deals  with  the 
category  coding  process  for  the  post-implementation  interviews.  During  this  process, 
new  categories  had  to  be  created.  These  categories  were  not  present  for  the  first  series  of 
interviews  and  were  based  on  the  “themes”  that  evolved  in  the  second  set  of  interviews. 

The  first  category,  “the  purchase  order  process  is  better  than  it  was,"  and  the 
second  category,  “the  new  capabilities  and  information,  not  previously  available,  is  a 
strength  of  the  system”  are  important.  They  are  so,  not  only  because  they  are  the 
respective  modes  for  each  question,  but  because  they  represent  a  recurring  positive  issue 
that  was  absent  from  the  first  interviews.  The  following  direct  quotations,  from  the 
second  interviews  and  these  respective  categories,  will  add  further  to  this  line  of  thought 
and  illustrate  some  of  the  opinions  of  the  implementation.  The  first  quotation,  “It’s  easier 
to  get  stuff  than  it  used  to  be”  was  made  by  subject  JR.  In  its  context,  the  subject  was 
referring  to  her  ability  to  obtain  items  through  the  new  procedures.  The  implication  was 
that  the  new  procedures  have  made  this  task  easier.  The  second  quote,  “I  personally  have 
benefited  from  the  implementation,  the  better  procedures  have  made  my  job  easier”,  was 
made  by  subject  JE  when  she  was  referring  to  her  ability  to  quickly  obtain  purchase  order 
numbers  (her  job  productivity  and  effectiveness  was  increased  with  this  new  ability). 
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The  third  quotation,  “We  now  know  what’s  going  on,  there’s  no  more  guessing”, 
was  made  by  subject  LD,  and  was  one  of  the  statements  that  was  coded  for  the  second 
category  (identified  earlier).  In  its  context,  the  subject  was  referring  to  her  ability  to 
follow  the  purchase  order  process.  The  new  information,  that  before  was  not  available, 
now  allows  concurrent  monitoring  and  tracking  of  purchase  orders.  This  capability  is 
essential  to  the  operation  of  the  purchasing  department  and  is  more  evidence  of  perceived 
improvements  and  the  shift  in  opinions.  The  last  quotation,  “We  have  a  lot  more  options 
now,"  was  also  made  by  subject  LD,  and  was  referring  to  the  new  reporting  capabilities. 
These  capabilities,  which  were  nonexistent  before,  are  now  convenient,  simple  to  access, 
and  very  beneficial  to  the  organization. 

Continuing  with  the  questions  (on  the  coding  and  tally  sheet)  that  addressed 
knowledge  of  the  process  (Q5-Q8),  it  was  hoped  that  there  would  be  a  demonstrable 
difference  in  the  pre  and  post  interview  responses.  Although,  this  was  not  the  case,  there 
were  some  differences  that  warrant  discussion.  First,  the  difference  in  the  knowledge  of 
the  approval  process  was  noticeable.  This  should  be  attributed  to  the  success  of  the 
signature  authority  policy  implementation.  Anecdotal  evidence  suggests  that  this 
implementation  went  particularly  well.  Secondly,  the  average  time  for  receipt  of  ordered 
items  dropped  from  8.75  days  to  6.4  days.  This  may  be  attributed  to  the  new 
departmental  procedures  and  performance  goals,  or  it  may  not.  Direct  observations 
suggest  influences  attributed  to  the  Hawthorne  effect  may  have  played  a  role  in  this 
reduction.  A  combination  of  these  two  elements  is  more  likely  responsible  for  this 
reduction  in  receipt  time. 
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CONCLUSIONS 


This  project  has  accomplished  many  things,  they  are  loosely  grouped  into  two 
categories.  The  first  group  of  benefits  centers  around  information.  This  project  has 
dramatically  increased  the  amount  of  useful  information.  As  the  SMARRT  database 
continues  to  be  filled,  the  information  it  stores  will  be  used  for  historical  analysis,  trend 
analysis  and  determination  of  seasonal  pattems/cycles.  This  information  has,  and  will 
continue  to  be  used  to  track  cost  center  usage,  high  cost  items,  and  high  volume  items. 
This  in  turn,  will  enable  aggressive  contract  re-negotiations  and  solicitation  with  other 
vendors  for  prime  vendor  status  and  significant  cost  savings.  All  of  these  features  are 
new,  and  were  not  possible  with  the  old  system. 

The  second  group  of  benefits  centers  around  control.  The  old  process  suffered  from 
a  general  lack  of  control  and  this  project,  for  the  most  part,  has  alleviated  this  problem. 
There  have  been  many  initiatives  developed  and  implemented.  First  and  foremost,  the 
development  and  introduction  of  detailed  policy  guidelines  has  clarified  and  streamlined 
the  purchase  order  process.  Further,  the  new  policies,  and  their  methods  of  introduction, 
have  increased  the  general  understanding  of  the  processes  involved  and  reduced  the  level 
of  associated  problems.  This  new  efficiency,  and  the  previously  mentioned  wealth  of 
information  has  been  the  result  of  the  implementation  of  a  purchase  order  tracking  system 
that  is  clearly  superior  to  its  predecessor.  This  has  been  done  to  the  benefit  and 
satisfaction  of  the  manager,  the  customers  of  the  process,  and  to  the  organization. 


39 


REFERENCES 


Benbasat,  Melissa.  "The  Case  Research  Strategy  in  Studies  of  Information  Systems.” 
MIS  Quarterly  1 1,  no.  3  (Sept  1987):  369-86. 

Berry,  Valarie.  "The  Service-Quality  Puzzle."  Business  Horizons  31,  no.  5  (Sept/Oct 
1988):  35-43. 

Cook,  Thomas.  Qualitative  and  Quantitative  Methods  in  Evaluation  Research •  Beverly 
Hills:  Sage  Publications,  1979. 

Eliot,  Lance.  "Case  Analysis  of  Expert  Systems  Projects:  Strategies  and  Examples." 
Journal  of  Systems  &  Software  19,  no.  2  (Oct  1992):  153-57. 

Filstead,  William.  Qualitative  Methodology.-  Chicago:  Markham  Publishing,  1970. 

Ftiz-Gibbon,  Carol.  How  to  Analyze  Data.  Newbury  Park:  Sage  Publications,  1987. 

Furman,  Gail  Chase.  "Administrators'  Perceptions  of  Outcome-Based  Education:  A  Case 
Study."  International  Journal  of  Education  Management  9,  no.  6  (Jun  1995):  32- 
42.  * 

Kaplan,  Dennis.  "Combining  Qualitative  and  Quantitative  Methods  in  Information 
Systems  Reassert:  A  Case  Study."  MIS  Quarterly.  12,  no.  4  (Dec  1988):  571-86. 

King,  Jean.  How  to  Assess  Program  Implementation .  Newbury  Park:  Sage  Publications, 
1987. 

Miles,  Matthew.  Qualitative  Data  Analysis.  Beverly  Hills:  Sage  Publications,  1984. 

_ .  Qualitative  Data  Analysis :  An  Expanded.  Sourcebook.  Thousand  Oaks:  Sage 

Publications,  1994. 

Morris,  Lynn.  How  to  Communicate  Evaluation  Findings..  Newbury  Park:  Sage 
Publications,  1987. 


40 


_ .  How  to  Measure.  Performance  and.  Use  Tests.  Newbury  Park:  Sage 

Publications,  1987. 

Nachmias,  David.  Research  Methods  in  the  Social  Sciences .  New  York:  St.  Martin's 
Press,  1987. 

Patton,  Michael.  How  to  Use  Qualitative  Methods  in  Evaluation.  Newbury  Park:  Sage 
Publications,  1987. 

Stake,  Robert.  The  Art  of Case  Study,  Research.  Thousand  Oaks:  Sage  Publications,  1995. 

Tufte,  Edward.  The  Quantitative  Analysis  of Social  Problems.-  Reading:  Addison-Wesley 
Publishing,  1970. 

U.S.  General  Accounting  Office.  Evaluating  the  Acquisition  and  Operation  of 
Information  Systems ,  IMTD  TG2.  Washington,  D.C.,  1986. 

_ .  Content  Analysis:  A  Methodology  for  Structuring  and  Analyzing  Written 

Material  PEMD  10.1.3.  Washington,  D.C.,  1989. 

_ .  Case  Study  Evaluations.  PEMD  10.1.9.  Washington,  D.C.,  1990. 

_ .  Designing  Evaluations,  PEMD  10.1.4.  Washington,  D.C.,  1991. 

_ .  Using  Structured.  Interviewing  Techniques .  PEMD  10.1.5.  Washington,  D.C., 

1991. 

Van  Maanen,  John.  Qualitative  Methodology.  Beverly  Hills:  Sage  Publications,  1983. 

Yin,  Robert.  Applications  of  Case  Study  Research-  Newbury  Park:  Sage  Publications, 
1993. 

_ .  Case  Study  Research:  Design  and  Methods.  Thousand  Oaks:  Sage 

Publications,  1994. 


41 


